Rapid progress in the development of hand-held portable devices such as smartphones, palmtop computers, portable media players, personal-digital-assistant (PDA) devices and the like, has led to proposed inclusion of novel features and applications involving image processing. In such an application, namely image annotation or captioning, a user points a portable device towards a scene, e.g. a landscape, a building, a poster, or a painting in a museum, and the display shows the image together with superimposed information concerning the scene. Such information can include names, e.g. for mountains and habitations, people names, historical information for buildings, and commercial information such as advertising, e.g. a restaurant menu. An example of such a system is described in EP1246080 and in EP2207113.
Annotation information can be supplied to portable devices by servers in a wireless communication network. A corresponding functional configuration of a communication network with servers and portable devices here will be designated as an annotation system.
WO05114476 describes a mobile image-based information retrieval system including a mobile telephone and a remote recognition server. In this system, the image taken with the camera of the mobile phone is transmitted to a remote server where the recognition process is performed. This leads to high bandwidth needs for transmitting the image, and to a delay for computing the annotations in the server and transferring them back to the mobile phone.
Many annotation systems and methods include a step of comparing an image acquired by the annotating device with a set of reference images stored in a database. As actual viewing angle and lighting conditions can be different with respect to the images stored in the database, the comparison algorithm should remove the influence of these parameters.
A further, more sophisticated image annotation technique uses 3D reference models. Often, this involves a registration process, i.e. a process of spatially transforming a captured (or target) image to align with a reference 3D model. In the case of a building, for example, a 3D model of the object is stored in a reference database together with the details to be annotated. The 2D image acquired by the portable device is registered with this model, and if a match can be found, the object is recognized and the corresponding annotations are superimposed onto the 2D image.
Image annotation models based on 3D models have the advantage over 2D models of being less depending on the viewing angle. A single 3D model can be used as a reference for matching with a plurality of different 2D images captured from different locations and at different angles. However, building a collection of 3D models is a difficult and tedious process; this usually requires 3D or stereo cameras. Moreover, the process of registering 2D captured images with 3D models is time consuming.
It is therefore an aim of the present invention to solve or at least mitigate the above mentioned problems of existing augmented reality systems.